Beyond multimedia adaptation: Quality of experience-aware multi-sensorial media delivery

Multiple sensorial media (mulsemedia) combines multiple media elements which engage three or more of human senses, and as most other media content, requires support for delivery over the existing networks. This paper proposes an adaptive mulsemedia framework (ADAMS) for delivering scalable video and sensorial data to users. Unlike existing two-dimensional joint source-channel adaptation solutions for video streaming, the ADAMS framework includes three joint adaptation dimensions: video source, sensorial source, and network optimization. Using an MPEG-7 description scheme, ADAMS recommends the integration of multiple sensorial effects (i.e., haptic, olfaction, air motion, etc.) as metadata into multimedia streams. ADAMS design includes both coarse- and fine-grained adaptation modules on the server side: mulsemedia flow adaptation and packet priority scheduling. Feedback from subjective quality evaluation and network conditions is used to develop the two modules. Subjective evaluation investigated users' enjoyment levels when exposed to mulsemedia and multimedia sequences, respectively and to study users' preference levels of some sensorial effects in the context of mulsemedia sequences with video components at different quality levels. Results of the subjective study inform guidelines for an adaptive strategy that selects the optimal combination for video segments and sensorial data for a given bandwidth constraint and user requirement. User perceptual tests show how ADAMS outperforms existing multimedia delivery solutions in terms of both user perceived quality and user enjoyment during adaptive streaming of various mulsemedia content. In doing so, it highlights the case for tailored, adaptive mulsemedia delivery over traditional multimedia adaptive transport mechanisms

Quality of experience study for multiple sensorial media delivery

Traditional video sequences make use of both visual images and audio tracks which are perceived by human eyes and ears, respectively. In order to present better ultra-reality virtual experience, the comprehensive human sensations (e.g. olfaction, haptic, gustatory, etc) needed to be exploited. In this paper, a multiple sensorial media (mulsemedia) delivery system is introduced to deliver multimedia sequences integrated with multiple media components which engage three or more of human senses such as sight, hearing, olfaction, haptic, gustatory, etc. Three sensorial effects (i.e. haptic, olfaction, and air-flowing) are selected for the purpose of demonstration. Subjective test is conducted to analyze the user perceived quality of experience of the mulsemedia service. It is concluded that the mulsemedia sequences can partly mask the decreased movie quality. Additionally the most preferable sensorial effect is haptic, followed by air-flowing and olfaction.This work was supported in part by Enterprise Ireland Innovation Partnership programme

Region of interest-based adaptive multimedia streaming scheme

Adaptive multimedia streaming aims at adjusting the transmitted content based on the available bandwidth such as losses that often severely affect the end-user perceived quality are minimized and consequently the transmission quality increases. Current solutions affect equally the whole viewing area of the multimedia frames, despite research showing that there are regions on which the viewers are more interested in than on others. This paper presents a novel region of interest-based adaptive scheme (ROIAS) for multimedia streaming that when performing transmission-related quality adjustments, selectively affects the quality of those regions of the image the viewers are the least interested in. As the quality of the regions the viewers are the most interested in will not change (or will involve little change),the proposed scheme provides higher overall end-user perceived quality than any of the existing adaptive solutions

Perceived synchronization of mulsemedia services

Multimedia synchronization involves a temporal relationship between audio and visual media components. The presentation of "in-sync" data streams is essential to achieve a natural impression, as "out-of-sync" effects are often associated with user quality of experience (QoE) decrease. Recently, multi-sensory media (mulsemedia) has been demonstrated to provide a highly immersive experience for its users. Unlike traditional multimedia, mulsemedia consists of other media types (i.e., haptic, olfaction, taste, etc.) in addition to audio and visual content. Therefore, the goal of achieving high quality mulsemedia transmission is to present no or little synchronization errors between the multiple media components. In order to achieve this ideal synchronization, there is a need for comprehensive knowledge of the synchronization requirements at the user interface. This paper presents the results of a subjective study carried out to explore the temporal boundaries within which haptic and air-flow media objects can be successfully synchronized with video media. Results show that skews between sensorial media and multimedia might still give the effect that the mulsemedia sequence is "in-sync" and provide certain constraints under which synchronization errors might be tolerated. The outcomes of the paper are used to provide recommendations for mulsemedia service providers in order for their services to be associated with acceptable user experience levels, e.g. haptic media could be presented with a delay of up to 1 s behind video content, while air-flow media could be released either 5 s ahead of or 3 s behind video content

Quality utility modelling for multimedia applications for Android mobile devices

With the advances in mobile technologies, smart mobile computing devices have become increasingly affordable and powerful, leading to a significant growth in both the number of advanced mobile users and their bandwidth demands. Moreover multimedia streaming to these high-end mobile devices has become widespread. However, multimedia applications are known to be resource-hungry and in order to cope with this explosion of data traffic, operators have started deploying different, overlapping radio access network technologies. One important challenge in such a heterogeneous wireless environment is to ensure an Always Best Experience to the mobile user, anywhere and anytime. This paper proposes the Quality Utility, a realistic mapping function of the received bandwidth to user satisfaction for multimedia streaming applications. The Quality Utility is mapped to a Google Nexus One Android Mobile device and validated through objective and subjective tests

An innovative machine learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments

The latest advances in terms of network technologies open up new opportunities for high-end applications, including using the next generation video streaming technologies. As mobile devices become more affordable and powerful, an increasing range of rich media applications could offer a highly realistic and immersive experience to mobile users. However, this comes at the cost of very stringent Quality of Service (QoS) requirements, putting significant pressure on the underlying networks. In order to accommodate these new rich media applications and overcome their associated challenges, this paper proposes an innovative Machine Learning-based scheduling solution which supports increased quality for live omnidirectional (360◦) video streaming. The proposed solution is deployed in a highly dy-namic Unmanned Aerial Vehicle (UAV)-based environment to support immersive live omnidirectional video streaming to mobile users. The effectiveness of the proposed method is demonstrated through simulations and compared against three state-of-the-art scheduling solutions, such as: Static Prioritization (SP), Required Activity Detection Scheduler (RADS) and Frame Level Scheduler (FLS). The results show that the proposed solution outperforms the other schemes involved in terms of PSNR, throughput and packet loss rate

On the impact of wireless network traffic location and access technology on the mobile device energy consumption

In the context of wireless user’s increasing demands for better device power and battery management, this paper investigates some factors that can impact the power consumption on the energy consumption of mobile devices. The focus is on two factors when performing multimedia streaming: the impact of the traffic location within a WLAN; and the impact of the radio access network technology (WLAN, HSDPA, UMTS). The energy measurement results show that by changing the quality level of the multimedia stream the energy can be greatly conserved while the user perceived quality level is still acceptable. Moreover, by using the cellular interface much more energy is consumed (up to 61%) than by using the WLAN interface

A utility-based priority scheduling scheme for multimedia delivery over LTE networks

With the mobile networks migrating towards LTE-Advanced and all-IP networks, people expect to connect to the Internet anytime, anywhere and from any IP-connected device. Moreover, nowadays people tend to spend much of their time consuming multimedia content from various devices with heterogeneous characteristics (e.g., TV screen, laptop, tablet, smartphone, etc.). In order to support uninterrupted, continuous, and smooth video streaming with reduced delay, jitter, and packet loss to their customers, network operators must be able to differentiate between their offerings according to device characteristics, including screen resolution. This paper proposes a novel Utility-based Priority Scheduling (UPS) algorithm which considers device differentiation when supporting high quality delivery of multimedia services over LTE networks. The priority decision is based on device classification, mobile device energy consumption and multimedia stream tolerance to packet loss ratio. Simulation results demonstrate the benefits of the proposed priority-based scheduling algorithm in comparison with two classic approaches

Power-friendly access network selection strategy for heterogeneous wireless multimedia networks

Apart from the number and types of applications available to users of diverse devices with various characteristics, a highly relevant issue in current and future wireless environment is the coexistence of multiple networks supported by various access technologies deployed by different operators. In this context, the aim is to keep the mobile users “always best connected” anywhere and anytime in such a multi-technology multi-application multi-terminal multi-user environment. Multimedia streaming to battery powered mobile devices has become widespread. However, the battery power capability has not kept up with the advances in other technologies and it is rapidly becoming a concern. Since multimedia applications are known to be high energy consumers and since the battery lifetime is an important factor for mobile users, this paper proposes a network selection algorithm which bases its decision on the estimated energy consumption. The proposed solution enables the multimedia stream to last longer while maintaining an acceptable user perceived quality by selecting the least power consuming network